Confocal laser endomicroscopy (CLE) is an advanced optical fluorescencetechnology undergoing assessment for applications in brain tumor surgery.Despite its promising potential, interpreting the unfamiliar gray tone imagesof fluorescent stains can be difficult. Many of the CLE images can be distortedby motion, extremely low or high fluorescence signal, or obscured by red bloodcell accumulation, and these can be interpreted as nondiagnostic. However, justone neat CLE image might suffice for intraoperative diagnosis of the tumor.While manual examination of thousands of nondiagnostic images during surgerywould be impractical, this creates an opportunity for a model to selectdiagnostic images for the pathologists or surgeon's review. In this study, wesought to develop a deep learning model to automatically detect the diagnosticimages using a manually annotated dataset, and we employed a patient-basednested cross-validation approach to explore generalizability of the model. Weexplored various training regimes: deep training, shallow fine-tuning, and deepfine-tuning. Further, we investigated the effect of ensemble modeling bycombining the top-5 single models crafted in the development phase. Welocalized histological features from diagnostic CLE images by visualization ofshallow and deep neural activations. Our inter-rater experiment resultsconfirmed that our ensemble of deeply fine-tuned models achieved higheragreement with the ground truth than the other observers. With the speed andprecision of the proposed method (110 images/second; 85% on the gold standardtest subset), it has potential to be integrated into the operative workflow inthe brain tumor surgery.
展开▼
